Intraoperative gaze guidance with mixed reality.

Efficient communication and collaboration are essential in the operating room for successful and safe surgery. While many technologies are improving various aspects of surgery, communication between attending surgeons, residents, and surgical teams is still limited to verbal interactions that are prone to misunderstandings. Novel modes of communication can increase speed and accuracy, and transform operating rooms. A mixed reality (MR) based gaze sharing application on Microsoft HoloLens 2 headset that can help expert surgeons indicate specific regions, communicate with decreased verbal effort, and guide novices throughout an operation is presented. The utility of the application is tested with a user study of endoscopic kidney stone localization completed by urology experts and novice surgeons. Improvement is observed in the NASA task load index surveys (up to 25.23%), in the success rate of the task (6.98% increase in localized stone percentage), and in gaze analyses (up to 31.99%). The proposed application shows promise in both operating room applications and surgical training tasks.

Development of an augmented reality guidance system for head and neck cancer resection.

The use of head-mounted augmented reality (AR) for surgeries has grown rapidly in recent years. AR aids in intraoperative surgical navigation through overlaying three-dimensional (3D) holographic reconstructions of medical data. However, performing AR surgeries on complex areas such as the head and neck region poses challenges in terms of accuracy and speed. This study explores the feasibility of an AR guidance system for resections of positive tumour margins in a cadaveric specimen. The authors present an intraoperative solution that enables surgeons to upload and visualize holographic reconstructions of resected cadaver tissues. The solution involves using a 3D scanner to capture detailed scans of the resected tissue, which are subsequently uploaded into our software. The software converts the scans of resected tissues into specimen holograms that are viewable through a head-mounted AR display. By re-aligning these holograms with cadavers with gestures or voice commands, surgeons can navigate the head and neck tumour site. This workflow can run concurrently with frozen section analysis. On average, the authors achieve an uploading time of 2.98 min, visualization time of 1.05 min, and re-alignment time of 4.39 min, compared to the 20 to 30 min typical for frozen section analysis. The authors achieve a mean re-alignment error of 3.1 mm. The authors' software provides a foundation for new research and product development for using AR to navigate complex 3D anatomy in surgery.